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Abstract 

Acquiring sufficient linguistic proficiency to perform competently in academic and professional contexts generally 
requires substantial study time beyond what most language programs can offer in the classroom. As such, teachers 
and students alike would benefit considerably from high quality self-access materials promoting independent learning 
out of class when possible and, thus, allowing greater social interaction in class when necessary. The present study is 
a follow-up to Lyddon (in press), in which a concept-based approach to grammar instruction was adapted into a series 
of computerized modules. As the number of participants in the previous study was insufficient for meaningful 
statistical analyses of the learning outcomes, the current investigation redressed this shortcoming as well as 
responded to survey suggestions for several key changes to the pedagogical materials, including provision of more 
numerous training sentences, more highly elaborated feedback on exercises, and an immediate remediation option. As 
before, the purpose of the present study was to determine the viability of a computer-based conceptual approach to 
teaching English grammar. In a quasi-experimental pretest-posttest design, 3 1 Japanese university English learners of 
English for Science and Engineering completed a series of self-paced, interactive online tutorials aimed at teaching 
the concept of grammatical voice. Gains were then measured in terms of improvement in post-treatment 
grammaticality judgments. While the findings showed the modules to be generally effective, they also suggest 
important differences in learner understanding of the use of animate vs. inanimate subjects. Thus, additional study 
will be needed to focus on this particular issue. 
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1. Introduction 

While many university English programs are shifting their focus from general language use to language 
for specific purposes, the number of in-class instructional hours these programs can provide is often 
largely insufficient to foster the level of proficiency necessary for professional and academic success 
(Lyddon, 2011). Compounding this issue is the fact that some essential aspects of advanced language use 
are so difficult to acquire as to be thought virtually unteachable, as in the particular case of grammatical 
voice (cf. Hinkel, 2002). Drawing on Negueruela’s (2003) work on schemes for the complete operational 
basis of action, or SCOBAs, Johnson and Lyddon (2011) have fruitfully implemented a concept-based 
approach to teaching grammatical voice by targeting three of its core constituents, namely agency (i.e., 
the status of initiator of an action), animacy (i.e., the degree to which an entity is capable of willful 
action), and transitivity (i.e., the association between a verb and a direct object). The price of their 
success, however, was three full 90-minute lessons, or a total of 20 percent of the instructional time for 
the entire course. Lyddon (in press) previously attempted to explore the possibility of adapting a concept- 
based approach to the development of computerized modules that might serve as online self-access 
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materials to supplement face-to-face instruction and thereby save valuable in-class lesson time for other 
important pedagogical objectives. However, as limitations in the resulting data did not permit the desired 
statistical analyses, the current study redresses this issue as well as introduces a few suggested 
improvements in the instructional treatment. 

2. Method 

2.1. Procedures 

The study took place in three 60-minute sessions spanning a three- week period in the spring of 201 1 at a 
computer science university in northern Japan. The participants were 31 first- and second-year students 
who had been recruited on the basis of their TOEIC® reading scores, which ranged from 155 to 285, with 
a median of 200. Thus, although slightly above the 183 average for Japanese university English learners 
of the same age (Educational Testing Service, 2011), they still exhibited elementary- to intermediate-level 
reading proficiency and fell short of the 305 points generally equated with the level necessary to meet 
basic work requirements (Educational Testing Service Canada, n.d.). 

In the first session, the participants all completed a 30-item grammatical acceptability judgment task 
consisting of equal numbers of randomly ordered sentences representing five key categories. Moreover, to 
mitigate against potential differences in vocabulary difficulty, the past tenses of the same six regular 
verbs (i.e., analyze, damage, design, discuss, study, occur) were reused in combination with the same 
noun phrases in each category. For example, the various permutations for the verb ‘analyze’ were as 
follows: 

• Animate Agent + Active Transitive Verb (AAT) 

Ex.: We analyzed the results in two main ways. 

• Animate Agent + Passive Transitive Verb (APT) 

Ex.: We were analyzed the results in two main ways. 

• Inanimate Agent + Active Transitive Verb (IAT) 

Ex.: Two main ways analyzed the results. 

• Inanimate Agent + Active Intransitive Verb (IAI) 

Ex.: The results analyzed in two main ways. 

• Inanimate Agent + Passive Verb (IP) 

Ex.: The results were analyzed in two main ways. 

In the second and third sessions, the participants began the actual pedagogical treatment, which 
consisted of three computerized modules created in Adobe® Director® and designed to teach the 
concepts of agency, animacy, and transitivity through bilingual explanations of essential points and short 
closed-response type comprehension checks (e.g., items requiring the learner to identify the key elements 
of sentences) such as the one shown in Figure 1 . 

The first module opened with a rationale for learning to use grammatical voice, presented the concept 
of agency, and then concluded with a summary of the main points. The second module followed the same 
pattern, only starting with a review of agency and then focusing on the concept of animacy. The final 
module began with a review of both agency and animacy, introduced the concept of transitivity, and then 
took the learners through a metalinguistic probing exercise to help them interactively construct the 
constituent branches of a concept map to guide their thought processes in deciding when to use the active 
or passive voice, as illustrated in Figure 2. 

As each module took an average of approximately 25 minutes, the study participants completed the 
ones on agency and animacy in the second session and the one on transitivity in the third. Immediately 
after the conclusion of the transitivity module, they were given the same grammatical acceptability 
judgment task that they completed at the beginning of the study. 
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FIGURE 1 

Typical Comprehension Check Exercise 



FIGURE 2 

Completed Conceptual Map of the Decision-making Process for Choosing Active or Passive Voice 



2.2. Results 

To hedge against ceiling effects and lucky guessing, pretest scores above 80% were excluded casewise. 
Table 1 shows a comparison of the mean differences for the overall pretest and posttest scores as well as 
for performance on the five separate categories of test items. Two-tailed dependent samples t-tests at a .05 
alpha level indicated that the modules had an overall medium-sized positive effect on the learners’ ability 
to distinguish acceptable from unacceptable uses of grammatical voice: t( 17) = 2.562, p < .021, d = .6. 
However, this finding is mainly due to the large contribution of performance on the items involving 
animate subjects, for the posttest differences on those involving inanimate subjects were either non- 
significant (i.e., IAT and IAI) or undefined (i.e., IP, where the standard deviation was 0). 

Table 2, on the other hand, gives a more detailed look at the raw gain scores for individual item types, 
thus revealing that whereas the treatment had universally positive effects on recognition of acceptable use 
of voice with animate subjects, it produced a considerable amount of backsliding on items with inanimate 
subjects. 
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TABLE 1 

Comparison of Pretest-Posttest Score Differences 


Score 

Variable 

n* 

Mean 

Diff. 

Std. Dev. 

t 

df 

Sig. 

d 

Overall total 

18 

2.11 

3.50 

2.562 

17 

.020 

0.6 

AAT 

7 

1.43 

0.79 

4.808 

6 

.003 

1.8 

APT 

9 

1.89 

0.60 

9.430 

8 

< .001 

3.1 

IAT 

27 

0.37 

1.52 

1.264 

26 

.218 

0.2 

IAI 

16 

0.44 

2.00 

0.875 

15 

.395 

0.2 

IP 

5 

1.00 

0.00 

— 

— 

— 

— 


includes only scores less than 80% of max. possible on each relevant pretest measure. 


TABLE 2 

Raw Pretest-Posttest Gain Scores by Item Type 


Item Type 

Gain 

AAT 
(n = 7) 

APT 
(n = 9) 

IAT 
(n = 27) 

IAI IP 

(n = 1 6) (n = 5) 

+4 



1 

1 

+3 


1 

1 

1 

+2 

4 

6 

5 

3 

+ 1 

2 

2 

4 

3 5 

0 

1 


7 

4 

-1 



7 

1 

-2 



2 

1 

-3 




2 


3. Discussion 

The improved performance on items involving animate subjects is not surprising given the similarities 
between these types of sentences in Japanese and English, which also explain why the majority of 
participants scored over 80% for these item types on the pretest. Though disappointing, the mixed 
performance on the items involving inanimate subjects, both with transitive and intransitive verbs, is also 
understandable as it is in this area that the use of voice in Japanese and English most differs. Consistent 
with Lyddon (in press), some learners tended to overgeneralize the acceptability of active transitive verbs 
with inanimate subjects. In the case of active intransitive verbs, on the other hand, learner performance 
appeared almost random. One possible explanation for these findings is that the concept of inanimate 
agents may need further elaboration or clarification as it may not be obvious what they can and cannot do 
(e.g., a software application can produce a graph, but it cannot study one). 

Another possibility is that some learners are exhibiting U-shaped learning (Rumelhart & McClelland, 
1986; Pinker & Prince, 1988), that is, they are overgeneralizing previously acquired forms. A classic 
example of this phenomenon is LI English learners’ tendency to replace correct irregular past tense forms 
with non-existent regular forms (e.g., ‘went’ with ‘goed’) once they become cognizant of the rule for 
regular past tense formation. In short, knowledge of the acceptability of inanimate subjects with active 
transitive verbs in some instances may have led some learners to accept sentences they previously 
rejected. 

Still another possibility is that the entire conceptual model requires further elaboration. For instance, 
the current modules present only simple sentences comprising a single independent clause. However, the 
phrase ‘the effects studied in a computer simulation’, while incorrect as a complete sentence, would be 
perfectly acceptable in a sentence such as ‘The effects studied in a computer simulation were more clearly 
understood than those yielded though mere mathematical modeling.’ In fact, such use of passive verbs 
with copula deletion in dependent clauses is a common feature of scientific prose. Further complicating 
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this issue is the somewhat opaque nature of transitivity as many verbs can be either transitive or 
intransitive, depending on their actual use. 

Finally, and perhaps most obviously, the modules may need to include additional practice exercises to 
increase time on task. While it is understandable that the study participants may have rushed to complete 
the modules within the time allotted for the experiment, they spent an average of only 69.5 total minutes 
on them, compared to 125 in the previous study. 

4. Conclusions 

While the findings of the current study indicate that an automated approach to concept-based grammar 
instruction may be effective for at least some aspects of one particular pedagogical objective (in this case, 
the use of grammatical voice), it is clearly not a substitute for live teaching of it at this point as further 
exploration will be necessary to determine the cause of backsliding in some areas. A follow-up study 
using a think-aloud protocol with a purposive sample of the previous participants will hopefully provide 
insights into this issue as well as suggest additional improvements for the next round of module revisions. 
It must be kept in mind, however, that the ability to recognize correct forms does not necessarily entail 
the ability to produce them. As such, the effects of concept-based grammar teaching on actual production 
present another area that will require future study. 
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